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The MAILING DATE of this communication appears on the cover sheet with the correspondence address •• 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 
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- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 
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2a)D This action is FINAL. 2b)£<] This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 
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5) D Claim(s) is/are allowed. 

6) KI Claim(s) 1-21 is/are rejected. 
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application from the International Bureau (PCT Rule 17.2(a)). 
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DETAILED ACTION 



1 . This action is responsive to communications: amendment filed on 4/3/02 to the 
application filed on 11/20/98. 

2. Claims 1-21 are pending in the case. Claims 1, 10, 14, 18 are independent 
claims. 

3. The rejections of claims 1-5, 7-8, 10-13, 18-21 under 35 U.S.C. 103(a) as being 
unpatentable over Gasper in view of Yamada and Stelovsky have been withdrawn as 
pursuant to the applicants 1 argument. 

4. The rejections of claims 6 and 9 under 35 U.S.C. 103(a) as being unpatentable 
over Gasper, Yamada, and Stelovsky and further in view of Suzuki have been 
withdrawn as pursuant to the applicants 1 argument. 

5. The rejections of claims 14-15, 17 under 35 U.S.C. 103(a) as being unpatentable 
over Stelovsky in view of Yamada have been withdrawn in view of the applicants' 
argument. 

6. The rejection of claim 16 under 35 U.S.C. 103(a) as being unpatentable over 
Stelovsky in view of Yamada as applied to claim 14 above, and further in view of 
Gasper has been withdrawn in view of the applicants' argument. 



Priority 

7. Receipt is acknowledged of papers submitted under 35 U.S.C. 1 19(a)-(d), which 
papers have been placed of record in the file. 
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Claim Rejections - 35 USC § 103 

8. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

9. This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was 
not commonly owned at the time a later invention was made in order for the examiner to 
consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 103(a). 

10. Claims 1-21 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Sato (JP-8-293039A, 1 1/5/96, filed 4/24/95) in view of Ohba (JP-3-216767A, 9/21/91 , 
filed 1/21/90), the IDSs submitted by Applicants. 

Regarding independent claim 1, Sato discloses: 

- providing music control information and a synchronization signal in 

correspondence with the music to be played (abstract, possible to generate from 
music the motion of a human dancing to the music; section [0002], each part of 
music is made to correspond to the motion of images having basic shape; 
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section [0006] musical information and motions of the articulated object are 
applied to the image ....to display an animated image; section [0008]; section 
[0035] motion of a human dancing to the music) 

- generating a sound in accordance with the music control information to thereby 
play the music (section [0006], a sound buildup point in accordance with the 
musical information and the motions of the animated image) 

- generating a motion image of the object in matching with progression of the 
music, and utilizing the music control information to further control the motion 
image in association with the played music (sections [0014] and [0015], the 
motions of limbs of the articulated object is matched with the chord ) 

Sato does not disclose the motion parameters to determine the movement of movable 
parts of the object and utilizing the motion parameters to control the motion image. 
Ohba discloses: 

- generating parameters to determine the movement of movable parts of the object 
(pages 1 and 2, as shown in figures 2 and 3.... parameter Q associated with the 
motions of legs, parameter R associated with the motions of arms, and 
parameter S associated with the display position of the character) 

- utilizing the motion parameters to control the motion image (page 2, transform 
the shapes of respective parts of the character expressed with the shape data in 
response to the values of the parameters Q, R and S) 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Ohba into Sato since Ohba provides the motion 
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parameters of the movable parts of the character as well as the correspondent motions 
of these parts and Sato provides the synchronization of the movement of the body of a 
human to the music. The combination of Ohba and Sato would provide an effective way 
to control the synchronization of the motions of the movable parts of an object with the 
play of music by using parameters set for each motion of the body. 

Regarding claim 2, which is dependent on claim 1, Sato discloses preparing a frame of 
the motion image in advance to generation of the sound corresponding to the same 
data block to generate the prepared frame timely when generating the sound according 
to the same data block used for preparation of the frame (section [0014], ...chords 
detected with the chord detecting section are matched in advance with the motion data 
stored in the motion database..., motion output section visualizes to display the motion 
data generated...). 

Sato does not disclose analyzing a block of the performance data to perform said 
preparing. However, it would have been obvious to one of ordinary skill in the art at the 
time of the invention was made to have modified Sato to include analyzing a block of 
the performance data into Sato since the block of performance data is considered 
equivalent to the multimedia segment in Sato where the motion video segments, the 
time segments and the sound segments are generated to match with each other. 
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Regarding claim 3, which is dependent on claim 1, Sato discloses generating the key 
frames of the motion image in response to the synchronization signal according to the 
motion parameters and the music control information, and generating the sub frames 
inserted between the successive key frames by interpolation to smoothen the motion 

image (section [0027], generating M frames per one motion motion data for display 

are obtained by interpolating a motion data of m-th frame and a motion data of (m+1)th 
frame... generate motions according to the music) 

Regarding claim 4, which is dependent on claim 1, Sato does not disclose generating 
the motion image of an object representing an instrument player and analyzing the 
audio control information to determine a rendition movement of the instrument player for 
controlling the motion image as if the instrument player plays the music. 
Instead Sato discloses converting music into images by using the stored motions of an 
articulated object and the chord for generating the motion of a human dancing to the 
music (abstract; sections [0033] and [0035]). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have modified Sato to include that the motion object is an instrument 
player for performing the music since a music player is considered equivalent to a 
motion object with movable parts moving according to the music. 

Regarding claim 5, which is dependent on claim 1, Sato does not disclose generating 
the motion image according to the motion parameters to determine the movement of the 
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movable parts of the object with respect to the default positions of the movable parts, 
and resetting the motion image to revert the movable parts to the default positions in 
matching with the progression of the music. Instead Sato discloses the matching of 
motions of a human dancing to the music (abstract; sections [0005] and [0035]). 
Ohba discloses generating the motion image according to the motion parameters to 
determine the movement of the movable parts of the object with respect to the default 
positions of the movable parts (pages 1 and 2, parameter Q associated with the motions 
of legs, parameter R associated with the motions of arms, parameter S associated with 
the display position of the character... transform the shapes of respective parts of the 
character expressed with the shape data in response to the values of parameters Q, R, 



It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Ohba into Sata for facilitating the control of motions of an 
objects in accordance to the played music by using the set parameters for 
corresponding motions. 

Regarding claim 6, which is dependent on claim 1, Sato discloses that the 
synchronization signal is utilized to regulate a beat of music so that the motion image of 
the object is controlled in synchronization with the beat of the music (sections [0008] - 



s). 



[0011]). 
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Regarding claim 7, which is dependent on claim 1, Sato discloses the synchronization 
of the motion video and the music (abstract, sections [0027]-[0029]). However, Sato 
does not disclose specifying an instrument used to play the music. 
It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have modified Sato to include specifying an instrument used to play the 
music since it was well known that a music must be played by an instrument or a 
combination of some instruments. 

Regarding claim 8, which is dependent on claim 1, Sato discloses the synchronization 
of motion video and music (abstract, generating the motion of a human dancing to the 
music; section [0035]). Ohba discloses utilizing the motion parameters to control the 
motion image of the object (refer to claim 1 ). 

Sato and Ohba do not disclose controlling the amplitude of the sound to further control 
the motion image such that the movement of each part determined by the motion 
parameter is scaled in association with the amplitude of the sound. 
However, it would have been obvious to one of ordinary skill in the art at the time of the 
invention was made to have modified Sato and Ohba to include controlling the 
amplitude of the sound to further control the motion image such that the movement of 
each part determined by the motion parameter is scaled in association with the 
amplitude of the sound because of the following reason. The amplitude of the sound 
can be controlled by adjusting the volume of the sound. Since the movement of the part 
of the object is determined by a motion parameter, said movement is synchronized with 
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the music, and the amplitude of the sound can be adjusted, these features suggest to 
synchronize the movement of the motion images with the sound in a way that the 
motion images are adjusted to match with the up and down of the sound. In other 
words, the motion images are scaled in association with the amplitude of the sound. 

Regarding claim 9, which is dependent on claim 1, as mentioned in claim 6, Sato 
discloses that the dancer object, which is a motion object, and the music are 
synchronized (abstract, section [0035]). Ohba discloses the motion parameters 
correspondent to the movements of a motion object (pages 1-2), 

Claims 10-13 are for an apparatus of claims 1-4, and therefore are rejected under the 
same rationale. 

Regarding independent claim 14, Sato discloses: 

- sequentially providing performance data to perform the music and a timing signal 
to regulate progression of the music (abstract, possible to generate from music 
the motion of a human dancing to the music; section [0002], each part of music is 
made to correspond to the motion of images having basic shape; section [0006], 
musical information and motions of the articulated object are applied to the image 
....to display an animated image; section [0008]; section [0035], motion of a 
human dancing to the music) 
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- generating a sound in response to the timing signal and in accordance with the 
performance data to thereby perform the music (section [0006], a sound buildup 
point in accordance with the musical information and the motions of the animated 
image) 

- generating a motion image of the object to response to the timing signal to 
matching with the progression of the music and utilizing the performance data to 
modify the framework in association with the performed music (sections [0014] 
and [0015], the motions of limbs of the articulated object are matched with the 
chord; section [0005], generating timing to switch the motion of the articulated 
object from musical signal, detecting chord from musical signals, and selecting 
motions from motion database based on the chords in time to the timing to switch 
the motion) 

Sato does not disclose providing motion parameters to design a movement of the 
object, and generating a motion image comprises utilizing the motion parameters to 
form a framework of the motion image. 
Ohba discloses: 

- providing motion parameters for the movements of a moving pictures (pages 1 
and 2, as shown in figures 2 and 3.... parameter Q associated with the motions of 
legs, parameter R associated with the motions of arms, and parameter S 
associated with the display position of the character) 
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- utilizing the motion parameters to form a framework of the motion image (page 2, 
transform the shapes of respective parts of the character expressed with the 
shape data in response to the values of the parameters Q, R and S) 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Ohba into Sato to facilitate reproducing a motion object 
where the movement of the object is synchronized with the sound of the music by using 
the motion parameters for each movement of the motion object. 

Regarding claim 15, which is dependent on claim 14, Sato discloses: 

- generating a sound in response to the timing signal and in accordance with the 
performance data to thereby perform the music (section [0006], a sound buildup 
point in accordance with the musical information and the motions of the animated 
image) 

- generating a motion image of the object to response to the timing signal to 
matching with the progression of the music (section [0005], generating timing to 
switch the motion of the articulated object from musical signal, detecting chord 
from musical signals, and selecting motions from motion database based on the 
chords in time to the timing to switch the motion) 

- preparing a frame of the motion image in advance to generation of the sound 
corresponding to the motion image so that the prepared frame can be generated 
timely when the sound is generated according to the same block used for 
preparation of the frame (section [0014], motions of limbs of the articulated object 
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are stored in advance in the motion database, and the kinds of chords detected 
are matched in advance with the motion data stored in the motion database) 
Sato does not disclose analyzing a block of the performance data to perform said 
preparing. However, it would have been obvious to one of ordinary skill in the art at the 
time of the invention was made to have modified Sato to include analyzing a block of 
the performance data into Sato since the block of performance data is considered 
equivalent to the multimedia segment in Sato where the motion video segments, the 
time segments and the sound segments are generated to match with each other. 

Regarding claim 16, which is dependent on claim 14, Sato discloses generating the key 
frames of the motion image in response to the synchronization signal according to the 
motion parameters and the music control information, and generating the sub frames 
inserted between the successive key frames by interpolation to smoothen the motion 

image (section [0027], generating M frames per one motion motion data for display 

are obtained by interpolating a motion data of m-th frame and a motion data of (m+1)th 
frame... generate motions according to the music). 

Regarding claim 17, which is dependent on claim 14, as mentioned in claim 14, Sato 
does not disclose providing motion parameters to design a movement of the object 
representing a player of an instrument, and wherein the step of generating a motion 
image comprises utilizing the motion parameters to form the framework of the motion 
image of the player and utilizing the performance data to modify the framework for 
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generating the motion image presenting the player playing the instrument to perform the 
music. 

Instead, Sato discloses converting music into images by using the stored motions of an 
articulated object and the chord for generating the motion of a human dancing to the 
music (abstract; [0033], [0035]). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have modified Sato to include that the motion object is an instrument 
player for performing the music since a music player is considered equivalent to a 
motion object with movable parts moving according to the music. 

Claims 18-21 are for a machine readable medium of claims 1-4, and therefore are 
rejected under the same rationale. 

Response to Arguments 
1 1 . Applicant's arguments with respect to claims 1-21 have been considered but are 
moot in view of the new ground(s) of rejection. 

Applicants argue that Suzuki is not a proper reference due to the invalid priority. 
Examiner agrees. Suzuki is withdrawn from the rejections. 
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Applicants argue that Gasper, Yamada, and Stelowsky do not teach or suggest setting 
a basic parameter for a video image animation and then using music control to further 
control the movement of the video image. 

Examiner agrees. 

Sato discloses the correspondence between the movement of the movable parts of an 
articulated object and the played music (page 1 , abstract, the motion of a human 
dancing to the music; [0035]). 

Ohba discloses the motion parameters for producing an animation (page 1, parameter 
Q associated with the motion of legs, parameter R associated with the motion of arms, 
parameter S associated with the display position of the character; abstract, generate an 
animation in a real time fashion by generating parameters; page 2, parameters set 
according to timing). Ohba further discloses controlling the movement of the video 
image (page 2, transform the shapes of respective parts of the character expressed with 
the shape data in response to the values of the parameters Q, R, S). 
It would have been obvious to one of ordinary skill in the art at the time of the invention 
was made to have combined Ohba into Sato since both are drawn to producing an 
object with movable parts. The combination of Ohba into Sato would provide a better 
way to control the perform of the synchronization based the set parameters 
correspondent for each motion of the object as well as to include music as a sound 
component played in synchronizing with the motions of an object. 
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Conclusion 



12. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Robotham et al. (US Pat No. 6,160,907,12/12/00, filed 10/10/97). 
Intriligator (US Pat No. 6,163,323, 12/19/00, filed 4/3/98). 

Tadamura et al., Synchronizing Computer Graphics Animation and Audio, IEEE, 10/98, 
pages 63-73. 

Modler et al., Gesture Recognition by Neural Networks and the Expression of Emotions, 
IEEE, 10/98, pages 1072-1075. 

Lewis et al., Automated Lip-Synch and Speech Synthesis for Character Animation, 
ACM, 1987, pages 143-147. 

1 3. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Cong-Lac Huynh whose telephone number is 703-305- 
0432. The examiner can normally be reached on Mon-Fri (8:30-6:00). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Heather Herndon can be reached on 703-308-5186. The fax phone 
numbers for the organization where this application or proceeding is assigned are 703- 
746-7239 for regular communications and 707-746-7238 for After Final 
communications. 
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Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the receptionist whose telephone number is 703-305- 
9000. 
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